[GLUTEN-10621][VL] feat: Support cudf parquet connector#10622
Merged
jinchengchenghh merged 15 commits intoapache:mainfrom Sep 30, 2025
Merged
[GLUTEN-10621][VL] feat: Support cudf parquet connector#10622jinchengchenghh merged 15 commits intoapache:mainfrom
jinchengchenghh merged 15 commits intoapache:mainfrom
Conversation
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Sep 23, 2025
Summary: Gluten needs to link velox_cudf_parquet_connector, so it cannot be OBJECT Aligns with LocalFileSystem to extractFilePath https://github.com/facebookincubator/velox/blob/main/velox/common/file/FileSystems.cpp#L109-L121 Related PR: apache/gluten#10622 Resolves: #14730 Pull Request resolved: #14728 Reviewed By: zacw7 Differential Revision: D83066640 Pulled By: xiaoxmeng fbshipit-source-id: 8835a883b4c517b0097e3afc1c83ac3994a91f38
jinchengchenghh
added a commit
to jinchengchenghh/velox
that referenced
this pull request
Sep 24, 2025
…ubator#14728) Summary: Gluten needs to link velox_cudf_parquet_connector, so it cannot be OBJECT Aligns with LocalFileSystem to extractFilePath https://github.com/facebookincubator/velox/blob/main/velox/common/file/FileSystems.cpp#L109-L121 Related PR: apache/gluten#10622 Resolves: facebookincubator#14730 Pull Request resolved: facebookincubator#14728 Reviewed By: zacw7 Differential Revision: D83066640 Pulled By: xiaoxmeng fbshipit-source-id: 8835a883b4c517b0097e3afc1c83ac3994a91f38
jinchengchenghh
force-pushed
the
cudf_connector
branch
from
efde00a to
351ee0c
Compare
|
Run Gluten Clickhouse CI on x86 |
jinchengchenghh
force-pushed
the
cudf_connector
branch
from
eb864a0 to
c27155f
Compare
jinchengchenghh
force-pushed
the
cudf_connector
branch
from
c27155f to
63bead8
Compare
jinchengchenghh
force-pushed
the
cudf_connector
branch
from
7d46834 to
54ccb43
Compare
FelixYBW
approved these changes
Sep 29, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cudf connector supports parquet file format, and does not support partition.
The splitInfo partitionColumns should be empty for unpartitioned table, but for iceberg split, it may access the partition columns even if it is unpartitioned table, so still set the partitionColumns field, and check it if map in it is empty to check if it is cudf not supported partition column.
Test on local.